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Abstract 

Recent publications have described and applied a novel metric that quantifies the genetic 
distance of an individual with respect to two population samples, and have suggested that the 
metric makes it possible to infer the presence of an individual of known genotype in a sample for 
which only the marginal allele frequencies are known. However, the assumptions, limitations, 
and utility of this metric remained incompletely characterized. Here we present an exploration 
of the strengths and limitations of that method. In addition to analytical investigations of 
the underlying assumptions, we use both real and simulated genotypes to test empirically the 
method's accuracy. The results reveal that, when used as a means by which to identify individ- 
uals as members of a population sample, the specificity is low in several circumstances. We find 
that the misclassifications stem from violations of assumptions that are crucial to the technique 
yet hard to control in practice, and we explore the feasibility of several methods to improve the 
sensitivity. Additionally, we find that the specificity may still be lower than expected even in 
ideal circumstances. However, despite the metric 's inadequacies for identifying the presence of 
an individual in a sample, our results suggest potential avenues for future research on tuning this 
method to problems of ancestry inference or disease prediction. By revealing both the strengths 
and limitations of the proposed method, we hope to elucidate situations in which this distance 
metric may be used in an appropriate manner. We also discuss the implications of our findings 
in forensics applications and in the protection of GWAS participant privacy. 

1 Introduction 

In the recently pubhshed article "Resolving Individuals Contributing Trace Amounts of DNA to 
Highly Complex Mixtures Using High-Density SNP Genotyping Microarrays" [Tj, the authors de- 
scribe a method by which the presence of a individual with a known genotype may be inferred as 
being part of a mixture of genetic material for which marginal minor allele frequencies (MAFs), 
but not sample genotypes, are known. 
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The method [Tl is motivated by the idea that the presence of a specific individual's genetic 
material will bias the MAFs of a sample of which they are part in a subtle but systematic manner, 
such that when considering multiple loci, the bias introduced by a specific individual can be detected 
even when his DNA comprises only a small fraction of the mixture. More generally, it is well known 
that samples of a population will exhibit slightly different MAFs due to sampling variance following 
a binomial distribution; the genotype of the individual in question contributes to this variation, 
and so may be "closer" to a sample containing him than to a sample which does not. Based on this 
intuition, the article [1] defines a genetic distance statistic to measure the distance of an individual 
relative to two samples, summarized as follows: 

Consider an underlying population P from which two samples F (of size np) and G (of size 
no) are drawn independently and identically distributed (i.i.d.) [in these are referred to as 
"reference" and "mixture" respectively]. Consider now an additional sample Y; we wish to detect 
whether Y was drawn from G, versus the null hypothesis that Y was drawn from P independent of 
G and F. Given the MAFs fi and gi at locus i for F and G, respectively, and given the MAFs m for 
sample Y with yi E {0, 0.5, 1} (corresponding to homozygous major, heterozygous, and homozygous 
minor alleles) at each locus i, [1] defines the relative distance of sample Y from F and G at i as: 

Di{Y) = \yi- fi\- \yi - gi\ . (1) 

By assuming only independent loci are chosen and invoking the central limit theorem for the large 
number of loci genotyped in modern studies, the article [T] asserts that the Z-score of Di across all 
loci will be normally distributed, 

r(y) = 421^i^ = ^iii^ ~ iv(o, 1) (2) 

^Var(A)/s VVar(A)/s 

where (•) denotes the average over all SNPs i, s is the number of SNPs, and Eq. |2] exploits the 
assumption [1] that an individual who is in neither F nor G will be on average equidistant to both 
under the null hypothesis, i.e., /Uq = 0. The article [1] proposes using this approach in a forensics 
context, in which G is a mixture of genetic material of unknown composition (e.g., from a crime 
scene), and Y is suspect's genotype; by choosing an appropriate reference sample for group F, it is 
hypothesized that large, positive T will be obtained for individuals whose genotypes are included 
in G, and hence bias gi, while individuals whose genotypes are not in G should have insignificant 
T since they should intuitively be no more similar to the mixture sample G than they are to the 
reference sample F. 

In m , the authors applied this test to a multitude of individuals Y, each of which are present in 
the samples constructed by them for F or G, and report near-zero false negative rates. The article 
concludes that it is possible to identify the presence of DNA of specific individuals within a series 
of highly complex genomic mixtures, and that these "findings show a clear path for identifying 
whether specific individuals are within a study based on summary-level statistics." In response, 
many GWAS data sources have retracted the publicly available frequency data pending further 
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study of this method due to the concern that the privacy of study participants can be compromised. 
However, because no samples absent from both F and G were used, false positive rates — significant 
T for individuals neither in G nor F — are not assessed in practice; rather, they are simply assumed 
to follow the nominal false-positive rate a given by quantiles of the putative null distribution in 
Eq.|2] 

In this manuscript, we expand on [1 by investigating the method's robustness to several inherent 
assumptions: 

1. that F, G, and Y are all i.i.d. samples of the same population P and hence the difference of 
MAFs fi and gi in the two samples is small; 

2. that the loci i are independent, such that the central limit theorem may be invoked in Eq|2j 
and 

3. that an individual Y- in neither G nor F does not have sufficient genotype identity (e.g., via 
inheritance) to true positive individual 1+ that Di(Y-) ~ Di(Y+) for enough i to bias T(Y-). 

To investigate the effect of these assumptions, we begin with a statement of the problem that [1] 
attempts to address, analytically derive the effect of deviations from the assumptions, and empiri- 
cally explore the accuracy of the method in practice using real and simulated genotype data. We 
conclude with a discussion of the implications of our findings, both in forensics as well as regarding 
identification of individuals contributing DNA in GWAS. 

The results presented here reveal that membership classification via Eq. [2] is sensitive to the 
choice of F and G; that even a small amount of LD will alter the distribution of T for null samples; 
and that individuals who are related to members of F or G are frequently assigned significant T 
values. Our findings suggest that Eq. [2] will in practice yield a high false-positive rate if used to 
discern the membership of an individual in a specific sample, and when used for this purpose is 
likely be accurate only if the above assumptions are exceedingly well-met and the individual Y is 
believed a priori to be present in exactly one of F or G. However, although these findings suggest 
that Eq. [2]may have limited utility to reliably detect the identity of an individual in F or G without 
prior knowledge, it may be valuable for verifying that an individual is not in either sample, and we 
find some suggestion that the metric (Eq. [T| proposed in [1^ could perhaps be extended to other 
genetic-similarity problems (e.g., in ancestry inference). 

2 Materials and Methods 

We explore the performance of the method described in [Ij both analytically and empirically. For 
the empirical studies, we attempt to classify real and simulated samples into pools derived from 
publicly available data sources in order to assess the chances that an individual is mistakenly 
classified into a group which does not contain his specific genotype. The data used in these tests is 
described below: 
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2.1 Experimental genotypes and MAFs 

Real-world genotypes from publicly available data sets were retrieved as follows: 2287 samples with 
known genotypes were obtained from the Cancer Genomic Markers of Susceptibility (CGEMS) 
breast cancer study. The samples were sourced as described in [2]. Briefly, the samples comprised 
1145 breast cancer cases (sample group 0+) and a comparable number (1142) of matched controls 
(group C-) from the participants of the Nurses Health Study. All the participants were American 
women of European descent. The samples were genotyped against the Illumnina 550K arrays, 
which assays over 550,000 SNPs across the genome. To assess the genetic identity shared between 
samples, we computed the fraction of SNPs with identical alleles for all possible pairs of individuals; 
none exceeded 0.62. 

Additionally, 90 genotypes of individuals of European descent (CEPH) and 90 genotypes of 
individuals of Yoruban descent (YRI) were obtained from the HapMap Project [3j. In both cases, 
the 90 individuals were members of 30 family trios comprising two unrelated parents and their 
offspring. SNPs in common with those assayed by the CGEMS study and located on chromosomes 
1-22 were kept in the analysis (sex chromosomes were excluded since the CGEMS participants were 
uniformly female); a total of 481,482 SNPs met these criteria. 

Minor allele frequencies for case and control groups were computed from the CGEMS genotypes. 
Publicly-available minor allele frequencies from the 60 unrelated CEPH individuals were retrieved 
directly from the HapMap Project ^ . The distribution of MAF differences for each group may be 
seen in Fig. [1} 

2.2 Simulated Genotypes I 

To explore the potential for a sample whose genotype is drawn on /j or gi (without being a member 
of F or G) to be misclassified, five sets of 320 simulated genotypes were created by drawing a 
genotype for each SNP independently as a pair of Bernoulli trials from given allele frequencies: 

S.l: For each locus in each sample, genotypes were drawn on the CGEMS control allele frequencies 
for that locus. 

S.2: For each locus in each sample, genotypes were drawn on the CGEMS case allele frequencies 
for that locus. 

S.3: For each locus in each sample, genotypes were drawn on the HapMap CEPH [3j allele fre- 
quencies for that locus. 

S.4: For each sample, 50% of the loci were selected at random to have genotypes drawn on CGEMS 
case frequencies, and the other 50% had genotypes drawn on CGEMS control frequencies. 

S.5: For each sample, 50% of the loci were selected at random to have genotypes drawn on HapMap 
CEPH frequencies, 25% of the the of the loci were selected at random to have genotypes drawn 
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on CGEMS case frequencies, and the other 25% had genotypes drawn on CGEMS control 
frequencies. 

2.3 Simulated Genotypes II 

To further explore the influence of genetic similarity, two other simulation sets were created. Begin- 
ning with the MAFs from CGEMS controls, here denoted by pi, we create the first set as follows: 

1. Draw fi from Bin(2000,pj)/2000 to simulate the MAFs of a sample of 1000 individuals; 

2. Draw 1000 genotypes on Bin(2,pj)/2 to simulate genotypes of 1000 individuals who will 
comprise G; 

3. Construct 200 genotypes (Ys) for which q percent of SNPs are chosen at random to be 
identical to a specific G individual (selected at random for each of the 200 samples), and the 
other 1 — q fraction of SNPs are drawn on Bin(2,pj)/2; 

4. Perform step 3 for values of q in 0.01 increments from to 1, thus generating 100 pools of 200 
samples each who bear q identity to a true-positive individual, and apply Eqs. |l|2| to classify 
them against the F and G generated in steps 1 and 2. 

A second set is created as follows, also using the MAFs from CGEMS controls as pf. 

1. Draw fi, Qi independently from Bin(2000,pj)/2000 to simulate the MAFs of two samples of 
1000 individuals each; 

2. Draw 200 genotypes (Ys) on Bin(2,(l — q)pi + [q)g.i)/2 to simulate 200 individuals from a 
population with MAFs biased toward Ghy q percent; 

3. Perform step 2 for values of q in 0.01 increments from to 1, thus generating 100 pools of 
200 samples each to be classified against the F and G generated in step 1. 

By creating these sets, we ensure that we have samples for which all SNPs are independent in F 
and G, and that F and G are samples of the same underlying population; the classification can 
then be observed as a function of the similarity parameter q in both cases. 



2.4 Classification of real and simulated genotypes 

The method as described in [Ij and summarized in the Introduction was implemented using R [1] . 
Subsets of the real data (Sect. 2.1) and simulated data (Sect. 2.2) described above were classified 
in a total of 17 tests, starting with a total of 481,382 SNPs and excluding those which did not 
achieve a minor allele frequency >0.05 in both F and G for a given test. A summary of the tests is 
provided in Table [T} Additionally, a series of 200 tests using Y , fi, and gi as described in Sect. 2.3 
were performed. 
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3 Results 



We begin with an analytical exploration of the assumptions underlying Eq. T]2 followed by the 
results of the tests as described in Methods. 



3.1 Di and T under the null hypothesis 

To address the need for a fully rigorous examination of the problem which [1^ tries to address, we 
here attempt to set up an idealized situation to which the theory and methods in [1] apply, and 
consider the properties of Di and T (Eqs. [l] [2]) in that setting versus deviations from that setting. 

Let us assume an underlying population P with MAFs pi from which samples F (of size np) 
and G (of size no) are drawn i.i.d. Consider now an additional sample Y. The null hypothesis is 
that Y was drawn from P, independent of F and G; the alternative of interest is that Y is drawn 
from G (or, symmetrically, F). Under these idealized circumstances, we observe that: 

fi ~ B\n{2nF,pi)/2nF , (3) 
gi ~ Bm{2nG,p^)/2nG, (4) 
Vi - Bin(2,pi)/2, (5) 

where the factors of two are a consequence of each sample possessing two independent alleles per 
locus. In [1], it is proposed that T (the Z-score of Di across all SNPs) follows a standard normal 
distribution (Eqs. [Tpj ). This proposition rests upon two assumptions: namely, that the mean {Di) 
across all SNPs under the null hypothesis is zero, i.e., /Uq = in Eq. |2j and that the SNPs i are 
completely independent such that we can write the variance of the mean as the mean variance, ie, 
Var((Z)j)) = Var(Dj)/s in the denominator of Eq. |2} Below, we consider sources of deviation from 
T ~ A^(0, 1) under the null hypothesis. 

3.1.1 Deviations from = 

In the large-sample limit, under the null hypothesis, 

hm fi=pi; lim Qi = Pi , (6) 

and hence 

lim Di = lim (\yi - fi\ - \yi - gi\) = . (7) 

np ,nQ^oo np ,nQ^oo 

Intuition might further suggest that since fi and Qi are both drawn from binomial distributions 
which are symmetric about pi, any sampling deviations resulting from finite upjUc will fall sym- 
metrically, and hence = 0. As we will show below, however, this conclusion is sensitive to two 
assumptions: 

1. that the MAF differences between samples F and G, fi — gi are small; 
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2. that the sample sizes np and no are not only large, but comparable. 

Because the number of SNPs s is quite large, slight deviations away from /io = have the power to 
shift the location of the null distribution of T considerably, rendering T incomparable to a standard 
normal unless the true /xq is known. Consider that the difference in T with and without the = 
assumption is 

T - r„„=o = , ^° (8) 

and that because Di ranges on (—1, 1), max(Var(Dj)) = 2. This means that 

min(r - r^(,=o) = ^1^0 (9) 

which can be quite large for even small values of since the number of SNPs s is on the order of 
10^. It is thus essential that /io be known or controllable. 

Dependence of fio on slight differences in MAFs /j — gi. 

Let us begin by writing the difference between MAFs fi and gi at locus i as n, 

fi = gi + ri. (10) 

We can then write 

Di = \yi- gi -Ti\ -\yi- gi\ , (11) 

and thus 

^J■o = ilvi - gi - n\ - Ivi - gi\) , (12) 

where /Uq is {Di) under the null hypothesis. 

We next make a simplifying assumption: since pi are the minor allele frequencies and thus 
< Pi < 0.5, and since fi and gi are estimates of pi, with few exceptions we will have < < 0.5 
and < gi < 0.5 (eliminating this assumption does not significantly alter the results). Under this 
assumption we can write 



(13) 



and hence Eq. [12] may be written 

MO = ^[ri- = 0\p^) - n ■ = 0-5|k) - Ti ■ ¥{yi = l|pO] IP(p») n^i) , (14) 

i 

where P(-) denotes probability and where we have exploited the fact that because F, G are inde- 
pendent samples of P, Ti is independent of p,, i.e., P(rj|pj) = P(Ti). Observing that 

nvi = 0\p.i) = {1 - pif ; 

F{yi = 0.5\pi)=2pi{l-pi); (15) 
F{yi = l\pi) = pj , 







for 


Vi 


= 0; 


Vi- gi- Ti\ - \yi - gi\ = < 




for 


Vi 


= 0.5 






for 


Vi 


= 1. 
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Eq. [14] becomes 

/^o = 5^(1 - + 2pf) n ¥{pi) nn) (16) 

i 

= ((1 - Api + 2pj) Ti) , (17) 
which is readily verified by simulation. 



Eq. 17 implies that when Tj deviates from zero, either due to systematic differences in F and G 
(i.e., violation of the assumption that both are drawn on the same population P) or due to sampling 
variation, the location of the null distribution of the test statistic given by Eq. [2] will be shifted 
by an amount equal to ((1 — 4pi + 2pf)Ti) ■ ^ s/Var(L>j) relative to that under the assumption that 
/io = 0. It is important to note that the shift is a weighted average of r^; ie, it depends not only 
on the differences in MAFs Tj but also on pj, and hence it is not sufficient that (rj) = 0, since 
small Ti will be amplified when pi is small and reduced when pi is large. As a result, predicting the 
deviation away from /iq = to properly calibrate T requires knowing not only Ti = fi — gi, but pi 
as well. 

In practice, Tj is easily calculated (examples of the distribution of Tj for the CGEMS and HapMap 
CEPH groups are given in Fig. [T]). On the other hand, knowing pi requires making assumptions 
about the population from which Y is drawn. In the case where Y is, in fact, drawn from a different 
underlying population than are F and G, the pi are difficult to obtain from the given data and the 



shift in T resulting from Eq. 17 is not readily calculated. (This effect is revealed in the empirical 
tests shown in Fig. [4] discussed in the empirical results section |3.2.1 below, wherein the HapMap 
samples are shifted by differing amounts.) 

Dependence of fj,Q on sample sizes np and nc- 

The effect of deviations from the second assumption above is intuitively obvious: if no > np, 
G will better approximate the underlying population P and so will be closer on average to a future 
sample Y. The dependence is derived explicitly in the Appendix. 

We can demonstrate this effect by simulation, as shown in Fig. |2j Here, we begin by creating 
10^ SNP MAFs Pi uniformly distributed on the interval (0,0.5). From these pi, we simulate the gi 
with sample size uq = 1000 as given by Eq. |4] (i.e., a binomial sample) as well as 200 independent 
samples Y with yi as given by Eq. [5] By simulating fi per Eq. [3] as np is varied and computing 
(Di) for each sample Y per Eq. [T| we can observe the dependence of (Di) under the null hypothesis 
(i.e. /io) on the sample size of np. A plot of the result is provided in Fig. |2] As seen in the plot 
and derived explicitly in the Appendix, the dependence in this case varies indirectly with np; as 
expected based on the intuition above, smaller np leads to larger values of (Di), indicating that 
Y is closer to G (the larger, more representative sample of P) than it is to F. Although the 
difference is small, {Di) / ^J\f^r\T)i)Js - given in Fig. |2|b) - is quite large, which would lead to a 
high false-positive rate in practice if the /io = assumption were used and T values compared to 
the presumed null distribution A^(0,1). Thus, we see that as np decreases, the distribution of T 
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under the null hypothesis diverges from the standard normal distribution, resulting in a higher false 
positive rate than that predicted by the nominal a from the standard normal. 



3.1.2 Deviations from Var((A)) = Var(A)/s 

Invocation of the central limit theorem to compare T to a standard normal distribution (as given 
in Eq. |2]) requires that the variance of the mean of Di be estimable by the mean of the variance, ie, 
Var((Z)j)) = Var(Z?i) /s. This, in turn, requires that the Di are uncorrelated. However, if the various 
Di are correlated — most notably due to linkage disequilibrium — this is no longer true. Specifically, 
the variance of the mean for s variables Di with variance Var(Dj) and average correlation p amongst 
the distinct Di is given by 

Var((A)) = Q + Var(A) • (18) 

In the case where the average correlation amongst the Dj's is zero, Eq. [18] yields the result which is 
found in the denominator of Eq. |2j on the other hand, p ^ generates a (l + (s — l)p) multiplicative 
increase over the correlationless variance. The large number of SNPs s results in little room for 



any correlation between them: consider that Eq. 18 dictates that for a modest number of SNPs 
s = 5 • 10^ even a very slight average correlation between all pairs of SNPs p = 0.002 would result 
in a tenfold increase in Var(T); for 500K SNPs (s = 5 • 10^), p = 0.0002 causes a a two order of 
magnitude increase in Var(T). However, it is impossible to ascertain p simply from yi, fi, and gi. 
Instead, this issue may be addressed by choosing fewer SNPs and assuming that p is sufficiently 
small. 

3.2 Results of Empirical Tests 



To demonstrate the results derived in Sect. 3.1 above, as well as to explore the performance of the 



method in realistic situations, we carried out the computations described by Eqs. Tj2 for various 



F, G, and Y as described in Table [TJ Distributions of T for each of the 17 tests described in 
Table [T] are shown in the corresponding figures listed in the table. Bearing in mind the fact that 
\T\ > 1.64 yields a nominal a (p-value) of 0.05 and \T\ > 4.75 yields a nominal a = 10~^ when 
compared to a standard normal distribution, the vast majority of samples we tested which were 
in neither F nor G were misclassified as being members of one or the other group when using the 
a = 0.05 threshold for rejection of the null hypothesis; the misclassification rate was also higher 
than expected when using a nominal a = 10~^ threshold. The high false-positive rate in practice 
is attributable to sensitivity to the assumptions which underlie the method, as described above in 



Sect. 3.1 We present the results under the assumptions from and then discuss the possibility 



of improving them based on our analytical and empirical findings. 

3.2.1 Deviation from putative null distribution 
Choice of F and G. 
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In Sect. 3.1.1, we saw that T will depend on the characteristics of the samples F and G. The 
effect is demonstrated in the results shown in Fig.jsj In these plots, T statistics (Eq. [T][2]) are given 
for all the CGEMS and S.1-S.5 samples for three choices of F and G: 

• F = HapMap CEPH, G = CGEMS case; 

• F = HapMap CEPH, G = CGEMS control; 

• F = CGEMS control, G = CGEMS case. 

The distribution of minor allele frequencies for each of these three groups (CGEMS cases, controls, 
and HapMap CEPHs) and the distribution of MAF differences for all three pairs of these groups 
may be seen in Fig [T| Notably, even though it may reasonably be expected that the HapMap 
CEPH sample closely resembles the Caucasian subjects in CGEMS, the distributions of the allele 
frequencies is much more similar in CGEMS cases and CGEMS controls than in either group 
and HapMap CEPHs. (The most striking difference in the HapMap and CGEMS distributions 
occurs around 0.5, where it can be seen that the minor (MAF< 0.5) allele in the CGEMS samples 
sometimes has a frequency > 0.5 in HapMap CEPHs.) Importantly, the width of the the distribution 
of MAF differences Tj = fi—Qi is much greater when HapMap CEPHs are one of F/G: although the 
mean difference in allele frequencies is quite small (0.0003-0.001) in all cases, Var(rj) is an order of 
magnitude larger when HapMap CEPH is used as one of the the groups, leading to non-zero /iq via 



Eq. 17 Additionally, the sample size of the HapMap group is much smaller than that of CGEMS, 



thus biasing classification of an unknown sample toward the larger (and hence more representative) 



CGEMS sample when HapMap is used for one of the groups (cf. Sect. 3.1.1 and Appendix for 
associated derivations). 

As expected, using the HapMap CEPHs for F fails to separate the CGEMS case and control 
distributions, such that CGEMS controls and cases all yield high T (and hence would all be classified 
as cases) when G = CGEMS cases; the situation is analogous for G = CGEMS controls (Fig.|3| top 
and center left). Only in the situation where F and G have similar large sample sizes and similar 
MAFs (when G = CGEMS cases and F = CGEMS controls) is good separation achieved, with the 
T statistics generally falling on the appropriate side of (Fig. [3] bottom left); even so, 15 of the 
controls were misclassified as cases. This final case, which achieves 99.4% accuracy using \T\ > 1.64 
(nominal a = 0.05), is analogous to the data presented in jl], for which all samples are in either F 
or G. As anticipated, the accuracy of the classification of cases and controls is dependent on the 
choice of F and G. 



The classification of the 1600 samples described in Sect. 2.2 with the same choices of F and G 
(right column of Fig. [s]) is also instructive. In all three cases, all samples achieve high T statistics 
despite the fact that they are in neither F nor G, frequently with \T\ ^ 4.75, i.e., a nominal p- 
value less than 10"^. (No simulated sample genotype was identical to any true positive genotype at 
greater than 62% of loci, comparable to the degree of genetic identity observed in the real samples.) 
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That is to say, the method classifies as positive individuals who possess a genotype yi that is drawn 
on fi or gi, but who are not necessarily in F or G. This is unsurprising, since Eqs. lj2 quantify 
the degree to which Y is not equidistant from F and G. Furthermore, this suggests that relatives 



of true positives may be misclassified (we consider this below in Sect. 3.2.4). 

Classification of null samples when F and G are well-chosen. 

Having observed the sensitivity of the classifier to the appropriate choice of F and G, we now 
explore the classification of samples which are in neither F nor G in the case where F and G are 
well-chosen. Here, we randomly select 100 cases and 100 controls from CGEMS to form an out-of- 
pool test sample set comprising 200 individuals, and recompute the MAFs for the remaining 1045 
CGEMS cases (G) and 1042 CGEMS controls {F). (Several such random subsets were created; the 
results were consistent and hence we present a single representative one.) SNPs were kept subject 
to the same constraint (MAF> 0.05 in both F and G) as above, and T statistics (Eq. [T] |2]) were 
computed for all the test samples using and gi as described. 

For the positives samples (those in F oi G), the classifier performs fairly well, correctly classi- 
fying 2083 samples (and calling 4 as in neither F nor G). However, of the 200 test samples which 
were in neither F nor G, only 62 have \T\ < 1.64, and the bulk are misclassified into the reduced 
group of CGEMS cases. The rate of false positives is thus 69% if T is used as an indicator of group 
membership under the assumptions in \T] at the nominal a = 0.05 (see Table |2]). A plot of the T 
values for all samples is given in Fig. |4]^A). A similar test, in which HapMap individuals unrelated 
to the CGEMS participants (90 each from CEPH and YRI groups) were classified against the same 
subsets of 1045 CGEMS cases (G) and 1042 CGEMS controls (F), yields similar results: ah the 
YRI individuals and 85/90 of the CEPH individuals were misclassified into the group of CGEMS 
cases at a = 0.05; a plot of the T value distributions are given in Fig. |4]^B). Selecting a more 
stringent a = 10~^ (the minimum reported in [1]) results in a 29.5% false-positive rate amongst 
the 200 out-of-pool CGEMS samples, 72% false-positive rate amongst HapMap CEPHs, and 100% 
false-positive rate amongst HapMap YRIs. A summary of the specificity and sensitivities obtained 
in this test is given in Table |2] 

The reason for the high false-positive rates in practice despite the stringent nominal false positive 
rate is clear from the plots Fig. |4]^A,B): namely, it can be seen that the putative null distribution 
(light grey line, A^(0, 1), cf Eq. |2| does not correspond to the observed distribution for samples for 
which the null hypothesis is correct, with differences in both the location and width. 

The overall shift to the right is a product of the small differences in fi — gi which accumulate as 
given by Eq. [17} Because in this test we happen to know the MAFs pi along with fi and gi for each 



of the CGEMS samples, we can compute /io given by Eq. 17 as 1.133-10 ^ and verify that, when 
divided by the average ^/yar{Di)/s w 5.6 • 10"^ amongst the samples, the center of the observed 
null distribution will be at T ~ 2. Indeed, visual inspection of Fig. |4|A) shows that shifting each T 
distribution by -2 would result in F, G, and null-sample distributions which lie more symmetrically 
about r = 0. Note, also, that the HapMap CEPHs and YRIs are shifted by different amounts than 
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are the CGEMS samples, due to the fact that the pi's which underlie the HapMap samples differ 
from each other and from CGEMS. From this, we can see that samples Y which are not drawn on 
the same population as F and G may in practice have a high false positive rate. 

The effect of LD, derived in Sect. 3.1.2 is also seen in these examples. In Fig.|4]^B), we observe a 



narrower distribution of T for the HapMap YRI samples versus the Caucasian CGEMS participants 
and HapMap CEPHs (the Yoruban individuals, who come from an older population, have lower 
average LD). The same effect is observed by comparing the distribution of T for the simulated 
samples in Fig. |3] (for which each SNP was independently sampled and hence have artificially low 
LD) to those of real populations. 

3.2.2 Correcting for deviations from A^(0, 1) 

Although the empirical false-positive rates obtained the the tests described above are exceedingly 
high, the distributions of T obtained in Fig. |4]^A,B) are nonover lapping. Hence, one might expect 
that if one could appropriately calibrate the thresholds of T at which classification is made, the 
sensitivity and specificity of the test could be considerably improved. (Note that, in practice, one 
does not know where the true-positive F and G distributions of T lie; this requires the genotypes 
of the F and G individuals.) Two approaches may be taken toward calibrating classification 
thresholds for T: an analytical approach, based on the results in Sect. [XT] above; or an empirical 
approach, based on constructing a null distribution from available samples. As we will see, both 
these approaches pose substantial difficulties. 

Analytical approach. 

In order to correct for the deviations from A^(0, 1) analytically, we need to know both the 
location and width of the distribution of T in the non-ideal circumstances under which the test is 
being conducted. That is, we need to know deviations from /io = resulting from MAF differences 



fi — Qi and sample size differences oi np and uq (cf. Sect. 3.1.1 and Appendix), as well as the 



average correlation amongst SNPs p (cf. Sect. 3.1.2 Eq. 18). 



Let us first consider the result in Eq. 17 which shows that in practice will be a function 
of the MAF differences Ti = fi — gi as well as the MAFs pi of the population P of which y is a 
sample. If we are well-assured that F and G are large samples of the same population P and that 
Y is also a sample of that population, an average of fi and gi may be used to estimate pi (the yi, 
while necessarily drawn on pi, are too small a sample to be a good estimate) and thus obtain ^o- 
Results of this approach (for the tests shown in Fig. |4] and Table [2]) are given in Table |3] in which 



Pi was estimated as {no ■ gi + np • fi) / {nG + np) and /io was computed according to Eq. 17, A slight 
improvement in the performance of the method can be seen by comparing the first two columns of 
Table [2] to those of Table [3 

However, the assumption used to compute pi (i.e., that Y , F, and G are all i.i.d. samples of the 
same population P) is one on which the accuracy of the correction is strongly dependent; consider, 
for instance, that the /io ~ 1.133 • 10~^ obtained for the simulations in Fig. [4|A,B) and discussed 
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above will produce the appropriate shift T ?)to=o — 2 for the 200 CGEMS samples in Fig. |4|A) 
using this method, but will not centralize the HapMap T distributions in Fig. |4]^B) appropriately, 
because the /j and gi are not good estimates of the MAFs of the populations from which the HapMap 
samples are drawn. Applying this correction to the HapMap samples (equivalent to moving the 
HapMap T distribution two units to the left in Fig. |4]^B)) results in a misclassification rate of 86% 
(nominal a = 0.05) and 44% (nominal a = 10~^) for the HapMap CEPHs and continued 100% 
misclassification of all HapMap YRIs. It is thus essential that if the /io given by Eq. 17 is to be 
used, sound estimates of pi need to be obtained. When Y is not a sample of the same population 
as or G, estimates of pj are unobtainable from /j, gi and yi alone, and hence this correction relies 
upon the assumption that G, -F, and Y are well-matched. 

The second influence on /Uq, described in both Sect. |3.l!T] and the Appendix, is the effect of the 
sample sizes np and no- Here, corrections are readily made, provided the sample sizes of F and G 
are known. In a forensics context, where G is a sample of unknown composition, uq may not be 
known; on the other hand, in other contexts (such as when using case and control MAFs from a 
GWAS), sample sizes are known and readily adjusted for. (In this test, np ~ no ~ 1000, and the 
correction is negligible.) 

We also saw in Sect. 3.1.2 and Fig. |4]^B) that the distribution of T for null samples will depend 
on the degree of correlation between the SNPs. To accurately derive the width of the T distribution 
for null samples, one would need to either select SNPs that yield vanishingly small p or know the 
value of p with high accuracy for the population of which y is a sample so that it can be discounted. 
The latter option requires knowledge beyond the MAFs of F and G and the genotype of individual 
Y] namely, it requires multiple genotypes from the population P from which Y was drawn such that 
the average correlation p between SNPs can be computed; even with a collection of null genotypes, 
the computation of the average pairwise correlation for 10^ SNPs is a computationally unfeasible 
task. Rather, selecting fewer SNPs in order to reduce LD is a more workable solution; the results 
of this approach can be seen in Fig. |4];C,D) and in Table [2} Here, 50,000 SNPs were selected, 
uniformly distributed across of the 481,382 SNPs used in Fig. |4];A,B). 50,000 SNPs was shown 
in |l] to be a reasonable lower bound to detect at nominal a ~ 10~^ one individual amongst 1000, 
which is the concentration of true positive individuals in this test. 

As is clear from Fig. [4] reducing the number of SNPs narrows the distributions considerably, 
yet at the same time brings them closer together such that the crisp separation previously obtained 
is reduced. Using this method, we see that the 200 CGEMS samples now have a distribution closer 
to that of the putative null A^(0, 1) such that using a threshold of a = 0.05 yields an improved — yet 
still larger than nominal — 21% false-positive rate while maintaining a high 96.3% true positive rate. 
However, the misclassification rate is still over 50% for both HapMap samples, and improving these 
values requires compromising the sensitivity, a direct result of the overlapping T distributions for 
the G and HapMap samples. 

Finally, we can consider applying both the SNP reduction and the pQ correction applied above; 
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the results here are given in the final two columns of Table [3j Because F and G are well- matched 
and the correction given by Eq. 17 is slight in the case of these 50,000 SNPs, the correction 
happens to offer little improvement over that achieved by subsetting the SNPs. 

Empirical approach. 

Another potential approach to obtaining a correct null distribution is purely empirical, namely, 
collecting a set of presumed-nuU genotypes (called A^) which can be assumed to be drawn from the 
same population as y, and determining the distribution of T for the null samples N . However, 
once again the method's sensitivity to the assumptions are a source of error. 

To see this, let us once more return to Fig. |4j In these figures, vertical bars represent the 0.05 
and 0.95 quantiles of the 200 CGEMS (black), 90 HapMap CEPH (cyan) and 90 HapMap YRI 
(blue) T distributions. 

Let us first consider a situation in which we have fi and gi, along with an individual Y who 
is one of the 200 CGEMS samples not in F or G, but no other genotypes. We might reasonably 
turn to publicly available HapMap genotypes as our group N from which we construct an empirical 
null distribution from which we set thresholds. The lines in Fig. |4]^A,C) depict this case. Using 
thresholds obtained from the HapMap CEPH distribution (cyan lines) still incorrectly classifies half 
of the 200 CGEMS samples; the false positive rate is yet greater (and the true-positive rate smaller) 
when using the HapMap YRI distribution. These lines illustrate the importance of choosing for N 



a sample which closely resembles Y — as with the choice of F for a given G in Sect. 3.2.1 HapMap 
CEPHs are insufficiently similar to CGEMS to provide accurate results, despite the fact that both 
samples are Americans of European descent. 

The converse is true as well: if we have N, F, and G which are well matched — such as illustrated 
in Fig. [4|^B,D), in which A^, F, and G all come from CGEMS data — yet Y is not drawn from the 
same underlying population as N/F/G, the method will incorrectly classify Y; roughly a quarter 
of the HapMap CEPHs and the majority of HapMap YRIs lie outside the thresholds set by the 200 
CGEMS samples in Fig. |§B,D). Once again, this underscores the importance of the assumption 
that F, G, and Y are all i.i.d. samples of the same population P, and — if a sample A^ is being used 
to construct a null distribution empirically — it, too, must be an i.i.d. sample of P. 

Another empirical option is that of simulating genotypes from the fi and gi to simulate T 
under the alternative hypothesis, with the assumption that the null and alternative hypothesis 
T distributions do not strongly overlap. However, this method also requires that F and G are 
large and well-matched samples, since (as can be seen in the top- and middle-right graphs in 
Fig. [3]) poorly-matched F and G will not produce crisply separated distributions. Furthermore, 
the thresholds derived by this approach will relate not to the false-positive rate but rather to the 
false-negative rate, i.e., these thresholds would control the power of the test, and the specificity in 
practice will remain unknown. 

We have thus seen that small deviations from the assumptions that F, G, and Y are i.i.d. 
samples of the same population P can produce false-positive rates which greatly exceed those 
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predicted by the null hypothesis. Even when these sources of error were adjusted for, in our tests 
we still observed a false positive rate that was higher than expected, such that the false positive rate 
was never less than 20% in practice for a nominal false-positive rate of 5%, and never less than 13% 
at a nominal false-positive rate of 0.0001%. While the distributions of T for the F, G, and various 
Y samples were observed to be separate in Fig. |4] we find that calibrating the thresholds accurately 
in absence of genotype information for F and G is not obviously doable. More importantly, it is not 
clear that, once thresholds are chosen, the empirical specificity could be assessed without additional 
genotype information from subjects who are well-matched to G and Y . 

3.2.3 Positive predictive value of the method. 

The effect of the modest specificity — even in the best of cases described above — on the posterior 
probability that the individual y is in F or G is considerable, given that the prior probability 
is likely to be relatively small in most applications of this method. Let us consider the positive 
predictive value (PPV), which quantifies the post-test probability that an individual Y with a 
positive result (i.e., significant T) is in F or G. This probability depends on the prior probability 
that the individual is in F or G, i.e., on the prevalence of being a member of F or G. PPV follows 
directly from Bayes' theorem, and is defined as 

PPV = Seus.Presj 

Sens •Prev-F(l- Spec) (1 - Prev) ' ^ ' 

where the PPV is the posterior probability that Y is in F/G given a prior probability of Prev. We 
can write this equivalently in terms of the positive likelihood ratio LR+, 

Posterior odds = LR+ • Prior odds (20) 

A plot of PPV vs. prevalence is given in Fig. [5j Even with the best sensitivity (99.23%) and 
specificity (87%) obtained in our tests — that in which G, and Y were drawn on the same 
underlying population P, ^uq was accurately computed, and a nominal a = 10^^ was used as a 
threshold (cf. Table |3]) — the prior probability (prevalence) of Y being in F/G needs to exceed 54% 
in order to achieve a 90% post-test probability that the subject is in F/G. For a PPV of 99%, 
the prior probability needs to exceed 72% for any specificity under 95%, assuming the observed 
sensitivity of 99%. We thus see the strong need for prior belief that y is in F or G. 

The difficulty in assessing the (empirical) specificity of the test in absence of additional data 
makes the posterior probability difficult to ascertain since the false positive rate in practice is 



much greater than that given by the nominal false-positive rate a. Eq. 21 underscores this fact; 
referring once more to the best result in Tables [2| [3| consider that LRj^ at 87% specificity and 99% 
sensitivity is 7.6, versus 990000 if the nominal false-positive rate a = 10~^ were correct. For prior 
probability of 1/1000, the first case yields a posterior probability of 1.1/1000, while the second 
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yields a posterior probability of 998/1000. These differences, which are difficult to measure without 
additional, well-matched null sample genotypes and which depend strongly on the degree to which 
the assumptions underlying the method are met (consider the differences between the CGEMS and 
HapMap CEPH specificities in Tables^ pose a severe limitation on the utility of using 
to resolve y's membership in samples F or G. 

3.2.4 Classification of relatives 

We now turn to the classification of individuals who are relatives of true positives. As discussed 
above in Sect. 3.2.1 the results from simulations S.1-S.5 in Fig. |3] suggest that individuals who 



are genetically similar, but not identical to, the subjects in pools F and G, frequently exhibit high 
|r|. This effect can be investigated by using HapMap families, since we can reasonably expect that 
the children will bear a greater resemblance to their parents than their parents do to one another. 
Recalling that the HapMap pools consist of thirty individual mother-father-offspring pedigrees, we 
construct pools as follows: 

• F = Mothers from pedigrees 1-15 and fathers from pedigrees 1-15 

• G = Children from pedigrees 1-15 and fathers from pedigrees 16-30 

and then compute T for mothers and children from pedigrees 16-30 using the same SNP criteria 
as before. The results of these tests for both the CEPH and YRI pedigrees, given in Fig. [6] are as 
expected, with the children having a significantly higher distribution of T than the mothers; the 
T values for all the children were so large that p-values <C 10~^^ were obtained when comparing 
to A^(0,1). By contrast, 5/15 of the YRI mothers from pedigrees 16-30 and 10/15 of the CEPH 
mothers from pedigrees 16-30 yielded |r| > 1.64 (with distributions roughly centered about T = 0). 
The wider distribution amongst the CEPHS again reflects the effect of LD. In Fig. |6]we can see 
that the method has the power to resolve three groups: those in a group, those related to members 
of a group, and those who are neither (as the groups become bigger, and hence more homogeneous, 
we would expect the distributions to move closer together, as evidenced by the lower range of T 
for the CGEMS-based tests in Fig. |3|. Note, however, that without knowing the distribution of T 
for true positives (which necessitates knowing the genotypes of true positives) setting a threshold 
to distinguish between true positives and their relatives is not possible by any of the methods 
described above. 

In order to explore the effect of genetic similarity in a controlled, ideal situation for which F and 
G are known to be samples of the same underlying population and for which all SNPs are known 
to be independent (i.e., in the ideal situation in which the putative null distribution A^(0, 1) should 



hold), we carried out the simulations described in Sect. 2.3 In these simulations, the underlying 



population P was taken to have MAFs pi as given by the CGEMS controls; /«, gi, and yi were 



derived as described in Sect. 2.3 as binomial samples oi pi. 
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In the first of these simulations, the test samples were constrained to have a proportion q of 
SNPs identical to a true positive individual, with the remaining SNPs drawn on pj. A plot of the 
false positive rate, defined as the fraction of the 200 simulated samples that achieve significant 
|T| > 1.64 {a < 0.05), as the similarity parameter q is varied is shown in Fig. [Tj Once simulated 
samples exceeded 65% identity with a true positive individual, they universally achieve significant 
T, and significant values of T are found over half the time for simulated samples exceeding 60% 
identity. (It should be noted that of the real samples, no two had > 62% fractional identity.) 

In the second set of these simulations, the test samples were drawn from a weighted mixture of 
MAFs: 

Vi ~ Bin(2,pO/2, (22) 
P^ = {l-q)Pi + {q)9i, (23) 

i.e., the sample was drawn from MAFs p'^ which are q percent like G and (1 — q) like CGEMS 
controls (MAFs pi). By simulating 200 samples for various q, computing T for each sample using 
the simulated F and G, and counting the number of samples that achieve significant |T| > 1.64 
at a = 0.05, we can see how the false positive rate varies with the percentage of G. Results are 
given in Fig [7j The misclassification rate exceeds 50% for q = 0.05; at q = 0.1, all samples yield 
significant T. 

The misclassification of relatives follows directly from the method's premise. Eqs. T]|2 together 



answer whether individual's genotype yi is closer to sample G's MAFs gi than to sample F 's MAFs 
fi than would be expected by chance, and it is unsurprising that a relative of a true member of G 
would appear closer to G (via Eqs. T]2) than to F. 



Put another way, Y being a member of G is sufficient but not necessary for y^ to be closer 
(via Eq. [T| to gi than to ff, it is possible for other sources of genetic variation to cause yi to be 
closer gi than to fi. We can observe this by turning once again to Fig. |4]j^A,C), where the dashed 
red and green lines show that the not-in-G CGEMS cases had a distribution of T closer to the 
other CGEMS cases G, and the not-in-F CGEMS controls had a distribution of T closer to the 
other CGEMS controls F, indicating that small class-specific genetic differences can yield altered 
values of T. The erroneous inferential leap that significant T results from Vs presence in F or G 
is responsible for the misclassification of relatives as well as for misclassification of non-relatives in 
the previous examples. 

4 Discussion and Conclusions 

In this work, we have further characterized and tested the genetic distance metric initially proposed 
in [T]. This metric, summarized here by Eqs. lj2 quantifies the distance of an individual genotype 



Y with respect to two samples F and G using the marginal minor allele frequencies fi and gi 
of the two samples and the genotype yi. The article |lj proposes to use this metric to infer the 
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presence of the individual in one of the two samples, and the authors demonstrate the utility of 
their classifier on known positive samples (i.e., samples which are in either F or G) showing that in 
this situation their method yields classifications of high sensitivity. Our investigations reveal that 
while the sensitivity is quite high (correctly classifying true positives into groups F and G) the 
specificity is considerably less than that predicted by the quantiles of the putative null distribution 
A^(0, 1). As a result, Eqs. [T] |2] are severely limited in their utility for discerning y's presence in 
samples F or G. 

In this work we have shown that high T values, significant when compared against A^(0, 1), may 
be obtained for samples that are in neither of the pools tested under several circumstances: 

• when pools F and G are sufficiently dissimilar such that the differences in fi and gi dominate. 



as seen in Sects. 3.1.1 and 3.2.1 as well as the Appendix 



• when y is a sample of a different population than are F and G, as seen in Sect. 3.2.1 



when a small amount of average LD is present such that the putative null distribution in 
Eq. [2] does not hold (due to a violation of the CLT assumption of independence), as seen in 
Sects. |3T2] and |3XT| 



and when a sample is genetically similar, but not identical to, individuals comprising F 01 G 



(e.g., relatives of true positives), as seen in Sect. 3.2.4 



The high false positive rates in the first two cases result from assumptions underlying the putative 
null distribution which are not met in practice, specifically, that the individual Y along with 
samples F and G are all i.i.d. samples of the same underlying population P, and that the amount 
of correlation between all s SNPs is vanishingly small. As we saw in Sect. |3.2.1 and 3.2.2 these 



assumptions are difficult to meet; for instance, HapMap CEPH and CGEMS samples are sufficiently 
dissimilar that they introduce error in violation of the first assumption, despite the fact that 
both samples are Americans of European descent. Adjusting for deviations from the putative null 
distribution also requires making strong assumptions or obtaining additional information, as seen 
in Sect. [3221 

Additionally, the conclusion that high T values result from y's presence in G relies upon the 
questionable assumption that individuals in neither F nor G will be equidistant from both, resulting 
in false positives even when the other assumptions are met. For instance, similarly genotyped 
individuals (both relatives and simulated samples) are often classified into the same group despite 
the fact that the other assumptions were met (Sect. ??). Amongst non-relatives, even when the 
thresholds have been adjusted for violations of the above assumptions as in Sect. [3.2.2 Eqs. |1|2| 



produce misleading classifications at a rate that is considerably greater than expected (21% vs. 
nominal 5% and 13% vs. nominal 0.0001% in the best cases reported in Table[2|. The unpredictable 
false positive rate in practice, resulting from the difficulty in accurately calibrating the significance 
of T, results in a likelihood ratio (and hence post-test probability) that is also unpredictable, with 
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higher false positive rates yielding lower post-test probabilities. When the prior probability of y's 
presence in F or G is modest, strong evidence (i.e., high specificity) is needed to outweigh this 
prior, which was not achieved in our tests (Sect 3.2.4). On the other hand, when samples were 
known a priori to be in one of the groups F/ G, Eqs. T]2 correctly identify the sample of which the 
individual is part (Sect. 3.2.4). 

These findings have implications both in forensics (for which the method [1] was proposed) 
and GWAS privacy (which has become a topic of considerable interest in light of [T). We briefly 
consider each: 

Forensics implications. 

The stated purpose of the method — namely, to positively identify the presence of a particular 
individual in a mixed pool of genetic data of unknown size and composition — is difficult to achieve. 
In this scenario, we have gi (from forensic evidence) and a suspect genotype yi. To apply the method, 
we would need 1) to assume that Y and G are indeed i.i.d. samples of the same population P; 2) 
to obtain a sample F which is also a sample of the underlying population P, well-matched in size 
and composition to G; 3) to obtain an estimate of the sample size of G such that sample-size effects 
can be appropriately discounted; and 4) to assume that the p- values at the selected classification 
thresholds are accurate. We have seen in the Results section the sensitivity to the assumption 
that Y, F, and G all come from the same population, the sensitivity to the sample size of G, and 
the difficulties in calibrating thresholds; the high false-positive rates which result from even small 
violations of these assumptions make it exceedingly likely that an innocent party will be wrongly 
identified as suspicious; its is even more likely for a relative of an individual whose DNA is present 
in G. 

GWAS privacy implications. 

Here the scenario of concern is that of a malefactor with the genotype of one (or many) indi- 
viduals, and access to the case and control MAFs from published studies; could the malefactor use 
this method to discern whether one of the genotypes in his possession belongs to a GWAS subject? 
In this case, F and G are known to be samples of the same underlying population P (due to the 
careful matching in GWAS), and their sample sizes are large and known. However, the malefactor 
still needs 1) to assure that y is a member of this population as well (as shown by the poor results 
when HapMap samples were classified using CGEMS MAFs) and 2) to assume that the p-values 
at the selected classification thresholds are accurate. Additionally, the prior probability that any 
of the genotypes in the malefactor's possession comes from a GWAS subject is likely to be quite 
small, since GWAS samples are a tiny fraction of the population from which they are drawn. Even 
if the malefactor were able to narrow down the prior probability to one in three, a sensitivity of 
99% and a specificity of 95% is needed to obtain a 90% posterior probability that the individual is 
truly a participant. 

On the other hand, if the malefactor does have prior knowledge that the individual Y partic- 
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ipated in a certain GWAS but does not know Vs case status, Eqs. [T] [2] permit the malefactor to 
discover with high accuracy which group Y was in. Additionahy, in the case of a priori knowledge, 
the participant's genotype is not strictly necessary, since a relative's DNA will yield a large T score 
that falls on the appropriate F/ G side of null. 

Despite these limitations, we have found that the distance metric (Eqs. [T] [2| may still have 
forensic and research utility. It is clear from both our studies and the original paper [1] that the 
sensitivity is quite high; in the (rare) case that a sample has an insignificant \T\ < 1.64, it is 
very likely that Y is in neither F nor G. We can also see that genetically distinct groups have T 
distributions with little overlap (Fig. |4]), and so it may be worth investigating the utility of Eqs. T]2 
for ancestry inference. 

On this note, let us once more consider the quantity which Eq. [T] measures, namely the distance 
of Ui from fi relative to the distance of from gi. Referring to Fig. [s] (right column) and Fig. [4]^ A, C), 
we can see that samples Y which are more like those in sample G have a distribution that lies to 
the right of samples which are more similar to F, as expected; for example, in Fig. |4]|^A,C), the 
distribution of null (not in F, G) CGEMS cases (dashed red line) is shifted to the right with respect 
to the distribution of null CGEMS controls, as mig ht be expected from Eq. [T] i.e., the CGEMS 
case ys are closer to CGEMS case Gs than are the CGEMS control Ys. Although this difference is 
not statistically significant, one could imagine that it may be possible to select SNPs for which the 
shift is significant, i.e., a selection of SNPs for which unknown cases are statistically more likely 
to be closer (via Eq. [T]) to the cases in G and unknown controls are statistically more likely to be 
closer to the controls in F. In this case, a subset of SNPs known to be associated with disease may 
potentially be used with Eqs. [T] [2]to predict the case status of new individuals; conversely, finding 
a subset of SNPs which produce significant separations of the test samples may be indicative of a 
group of SNPs which play a role in disease. Because this type of application would use fewer SNPs 
and would involve the comparison of two distributions of T (cases ^ {F, G} vs. controls ^ {F, G}), 
it may be possible to circumvent some of the problems stemming from the unknown width and 
location of the null distribution described above; still, much work is needed to investigate this 
possible application. If successful, the metric proposed in [l], while failing to function as a tool 
to positively identify the presence of a specific individual's DNA in a finite genetic sample, may if 
refined be a useful tool in the analysis of GWAS data. 

Appendix: Dependence of fiQ on the sample size of F and G 

Consider (Di) (cf. Eq. [T]) under the null hypothesis assumptions that Y, F, and G are all drawn 
i.i.d. from the same underlying population P with MAFs pi. Writing the probability distribution 
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of Pi as 



(A) is given by 

(A) ={\yi - fi\ - \yi - gi\) = ilvi - fi\) - {\yi - gi\) 

\yi - fi\ ny^\pi) nfi\pi) P(k) dyi dfi dpi- 

\yi - gi\ F{yi\pi) F{gi\pi) F{pi) dyi dgi dpi 



(A-1) 
(A-2) 



where we exploit the fact that Y , F and G are independent of each other but depend on the 
underlying population MAFs. 



The dependence of the first (second) term in Eq. A-2 on np {nc) is derived as follows. First, we 
note that since each yi is two Bernoulli trials (two alleles) with probability pi, we have the following 
values of \yi — fi\ with probability ¥{yi\pi) for each allowable value of y^: 



lUi - fi 



\Pi) = { 



(A-3) 



fory. = l; 
{\0.5- M)-{2pi{l- Pi)) for = 0.5: 
^(/^)•((l-p^)') for y, = 0. 

Moreover, since each fi follows a binomial distribution of size 2nF (two alleles per person), we 
invoke the normal approximation to the binomial for values np > 10 with mean pi and variance 
Pi{l - Pi)/(2nF)- Hence: 

2nF{fi-pi)'^' 



m\pi) 



2np 



y 27rpj(l - p, 



exp 



2pi{l -pi) 
exp [-A\i{fi-pif] , 



where we introduce 

A.F,i = \/nF/{pi{l -Pi)) 
to simplify the notation. In consequence, the first term of Eq. |A-2 can be written: 

(1 - fi)ipi) + (|0.5 - h\){2p,{l-pi)) + {hm-P^?) 

Afa 



(A-4) 
(A-5) 

(A-6) 



TT 



exp 



-AF,i{fi - Pi 



idf.dpi (A-7) 



and the second term may be written analogously for G. The absolute value in Eq. |A-7 is dealt 
with by considering the fi > 0.5 and fi < 0.5 cases separately, i.e., treating Eq. A-7 as the sum of 
integrals 



/oo r /»oo 
-oo [Jo. 



(1 - fi){pf) + {fi - 0.5)(2p,(l - k)) + (/*)((1 - Pif) nfilPi) dfi+ 



+ 



(1 - f,){pj) + (0.5 - /.)(2k(1 - p.)) + if^m-P^f) m\P^)df, 



\ dpi (A-J 
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Expanding the polynomials in Eq. A- 8 and once more using Eg. |A-6 to simplify notation, we rewrite 
the above as 



F,i 
TT 



0.5 



0.5 



^— oo 

where Ci, C2, C3, and C4 are functions of pi but independent of ff. 



oh 



Ci = 1 - 2pi 
C2 = - Pi , 
C3 = 1 - + 2p2 , 
Ci=pi. 



'i)dpi (A-9) 



(A-10) 
(A-11) 
(A-12) 
(A-13) 



Performing the interior integration in Eq. A-9 yields 



vr 



-A|_,(0.5~p.)2' 



+ (Cap, + Ci) 



IT 



\ + 



+ {{Ci-C3)Pi + {C2-C\) 



7rerfc(^i7,i(0.5 - Pi)) 



2A 



FA 



dpi . (A-14) 



Expanding out the various Cs as well as Ap^i, we now have for the first term of {Di 



{pi{l - Pi)) 



2 / n^(0.5-p,)n ^ 



+ 2(l-pi) + (2K-l)erfc 



lnF{0.5-pi)^ 
^(1 - Pi) 



dpi , (A-15) 



which has an indirect dependence on np. Performing the same integration for the second term 
in Eq. A-2| yields analogous indirect nc dependence. As a result, when np < nc the first term 
is greater than the second, yielding (Di) > 0; in the limit np,nQ 00, this difference becomes 
smaller. 

The dependence is illustrated in Fig.|2]A.. Here, we assume a uniform distribution of on (0, 0.5) 
and construct 10^ piS for the underlying population P from which we draw, independently, a sample 
G of size ug = 1000 and 200 samples Y from which we estimate {Di) under the null hypothesis. 
Sample F is drawn i.i.d. from P with sample sizes ranging from np = 10 to np = 1000, permitting 
us to plot {Di) as np is varied. The simulation results are shown as circles, overlayed with a plot 



of Eq. A-2 using the result in Eq. A-15 and assuming the uniform distribution oi pi. The values for 



{Di) obtained from the simulation closely matches those derived from Eq. A-15 In Fig. |2p, the 
corresponding values of T are presented. 
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481,382 SNPs 


50,000 SNPs 




a = 0.05 


a = 10-6 


a = 0.05 


a = 10-6 


Sensitivity 


99.8% 


97.5% 


96.3% 


36.3% 


Specificity, 200 CGEMS 


31.0% 


70.5% 


79.0% 


99.5% 


Specificity, 90 HapMap CEPH 


5.5% 


27.7% 


45.5% 


100.0% 


Specificity, 90 HapMap YRI 


0.0% 


0.0% 


4.4% 


97.7% 



Table 2: Empirical sensitivity and specificity for the tests shown in Fig. [4] assuming /Uq = 0. 
Classification results are given for two different nominal false positive rates a = 0.05 and a = 10-6. 





481,382 SNPs 


50,000 SNPs 




Q = 0.05 


a = 10-6 


a = 0.05 


Q = 10-6 


Sensitivity 


99.90% 


99.23% 


97.36% 


31.09% 


Specificity, 200 CGEMS 


40.0% 


87.0% 


78.0% 


99.5% 


Specificity, 90 HapMap CEPH 


14.4% 


55.5% 


54.4% 


100.0% 


Specificity, 90 HapMap YRI 


0.0% 


0.0% 


7.7% 


100.0% 



Table 3: Empirical sensitivity and specificity for the tests shown in Fig. |4] using /io as given by 
Eq. 17 and assuming that Pi = {np ■ fi + nG ■ gi)/{np + nG)- Classification results are given for two 
different nominal false positive rates a = 0.05 and a = W~^. 
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0.0 0.1 0.2 0.3 0.4 0.5 0.6 -0.06 -0.04 -0.02 0.00 0.02 0.04 0.06 

MAF CGEMS case MAF - CGEMS control MAF 

Figure 1: Distribution of minor allele frequencies (left) and differences (right) in CGEMS cases 
vs HapMap CEPHs (top), CGEMS controls vs HapMap CEPHs (center), and CGEMS cases vs 
CGEMS controls (bottom) . Note that the distribution of MAF differences is much narrower when 
comparing CGEMS cases to controls (bottom) than when comparing either to HapMap CEPH. 
Only SNPs achieving frequencies of 0.05 or more were considered. 
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<Di> for simulated null samples 
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Figure 2: Observed {Di) and T values for simulated data with varying sample sizes of np under 
the //Q = assumption. In A, open circles represent the average {Di) for each simulation; the 



solid line is the theoretical {Di) based on numerical integration of Eq. A-15 In B, boxplots of 
the observed Ts for each simulation are given assuming = 0; box boundaries correspond to the 
0.25 and 0.75 quantiles, and whiskers indicate the 0.05 and 0.95 quantiles (T values outside those 
limits are shown as square points). Horizontal lines at T = (green), T = 1.64 (corresponding to 
a = 0.05, in amber), and T = 4.75 (corresponding to a = 10~®, in red) are shown for reference; 
note that for np < 600, at least 25% of null sanies yield significant T at the nominal a = 0.05. 
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F=HM, G=C+ 



F=HM, G=C+ 




■ S 1 

■ S2 

■ S3 

■ S4 
S5 



-40 -20 
T statistic 



F=HM, G=C- 



F=HM, G=C- 




■ S 1 

■ S2 

■ S3 

■ S4 
S5 



-1 — 

-60 



n — 

-40 



n — 

-20 



F=C-, G=C+ 



F=C-, G=C+ 




Figure 3: Distribution of T for real CGEMS samples (left column) and simulated samples S.1-S.5 
(right column) using F/G pairs as follows: top, F = HapMap CEPHs, G = CGEMS cases; center, 
F = HapMap CEPHs, G = CGEMS controls; bottom, F = CGEMS controls, G = CGEMS cases. 
Only SNPs achieving frequencies of 0.05 or more were considered. Note that |T| > 1.64 is significant 
at the nominal a = 0.05 level and |T| > 4.75 is significant at the nominal a = 10^^ under the 
putative null distribution. 
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CGEMS subsamples 

: 1042 CGEMS controls, G = 1045 CGEMS cases 



putative null, N{0,r 
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T statistic; 481 ,382 SNPs 
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T statistic; 481 ,382 SNPs 
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T statistic; 50,000 SNPs 



HapMap samples 

: 1042 CGEMS controls, G = 1045 CGEMS cases 



putative null, N(0,i; 

■ in G (CGEMS Ca) 

■ in F (CGEMS Co) 
HapMap CEU 

■ HapMap YRI 




T statistic; 50,000 SNPs 



Figure 4: Comparison of T distributions for true positive and negative samples vs. putative null, 
starting with 481,382 SNPs in (A,B) and 50,000 SNPs in (C,D). In all plots, true positive F (1042 
CGEMS controls) is shown as a solid green curve, true positive G (1045 CGEMS cases) is shown 
as a solid red curve, and the putative null A^(0, 1) is given as a thin grey curve. The dark and light 
grey regions represent the areas for which the null hypothesis would be accepted at a = 0.05 and 
a = 10-^, respectively. In plots (A,C), CGEMS test samples in neither F nor G (100 CGEMS 
cases and 100 CGEMS controls) are given by a heavy black curve. The CGEMS case and CGEMS 
control distributions within this group are shown as dashed red and green lines, respectively. In 
plots (B,D), T distributions are given for HapMap CEPHs (cyan) and YRIs (blue). Vertical lines 
mark the 0.05 and 0.95 quantiles of the negative CGEMS samples (black), HapMap CEPHs (cyan), 
and HapMap YRIs (blue). 
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0.2 0.4 0.6 0.8 1.0 
Prevalence (Prior) 




0.2 0.4 0.6 0.8 1.0 
Prevalence (Prior) 



Figure 5: Positive predictive value (PPV) as a function of prevalence and specificity given 99% 
sensitivity. In (A) , PPV is shown on the y axis and color corresponds to specificity. The black curve 
depicts the 87% sensitivity line — the best sensitivity obtained in the empirical tests in Tables |2] [sj 
In (B), PPV is shown by color, and the y axis corresponds to specificity. 
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Figure 6: Distributions of T for out-of-group samples who are related (red line) and unrelated 
(blue line) to individuals in G for HapMap YRI (A) and HapMap CEPH (B) populations. (C) 
and (D) show the same distributions as (A) and (B) respectively, with the addition (green line) of 
individuals who are in G and unrelated to F (i.e., true positives). Dashed black lines indicate the 
T significance thresholds of ±1.64 at nominal a = 0.05. 



31 



^ Misclassification by fraction of SNPs identical to true positive 
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Figure 7: Misclassification rates for samples resembling true positives, as described in Sects. 2.3 



In (A), samples were generated which had fractional genotype identity to a specific true positive; 
the false positive rate is given as a function of the pairwise similarity. In (B), samples drawn on 
a distribution that is a proportional mixture of gi and the reference population MAFs; the false 
positive rate is given as a function of the proportion of gi. 
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